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Abstract;  We  consider  the  problem  of  optimal  stochastic  scheduling  for  nonlinear  systems  with 
Poisson  noise  disturbances  and  a  performance  index  including  both  operating  costs  and  costs  for 
sceduling  changes.  In  general,  the  value  functions  of  the  dynamic  programming,  quasi-variational 
inequalities  which  define  the  optimality  conditions  for  such  problems  are  not  differentiable.  How¬ 
ever,  we  can  treat  them  as  “viscosity  solutions”  as  introduced  by  Crandall  and  Lions.  Existence 
and  uniqueness  questions  are  studied  from  this  point  of  view. 
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1.  Introduction 


Optimal  scheduling  problems  arise  in  many  contexts,  including  inventory  control 
systems  and  unit  commitment  scheduling  in  electric  power  systems.  These  problems 
typically  involve  stochastic  dynamical  systems,  admitting  discrete  state  transitions  at 
random  times  as  control  actions,  and  incurring  both  switching  costs  and  continuous  run¬ 
ning  costs.  Using  the  dynamic  programming  principle,  one  can  show  that  the  optimality 
conditions  for  these  problems  are  expressed  mathematically  by  quasi-variational  inequali¬ 
ties  (QVI).  It  is  difficult  to  treat  QVI’s  explicitly,  and  most  of  the  work  has  focussed  on 
proving  existence,  uniqueness,  and  regularity  of  solutions. 

In  our  case,  the  state  system  is  forced  by  Poisson  noises.  Since  the  infinitesimal 
generator  of  the  state  process  is  first  order  and  has  a  translation  in  the  argument,  the 
associated  QVI  is  first  order  and  fully  nonlinear;  and  so,  the  standard  existence  and 
uniqueness  theory  developed  for  diffusion  -  parabolic  systems  does  not  apply.  To  treat 
the  problem,  we  use  the  method  of  viscosity  solutions  introduced  by  M.  G.  Crandall 
and  P.  L.  Lions  [1];  see  also  P.  L.  Lions  [2].  Various  properties  of  viscosity  solutions 
are  developed  in  Crandall  -  Evans  -  Lions  [3],  We  use  the  approach  in  Capuzzo  Dolcetta 

-  Evans  [4]  developed  for  deterministic  systems1.  We  prove  that  the  value  function  u 
associated  with  the  optimization  problem  is  a  viscosity  solution  of  the  corresponding 
(QVI).  Existence  of  solutions  to  the  (QVI)  is  shown  by  using  a  discrete  approximation  to 
an  associated  penalized  system  and  then  using  results  for  accretive  operators  as  in  [7]. 
On  the  other  hand,  we  use  dynamic  programming  to  obtain  a  decreasing  sequence  of 
value  functions  optimal  for  controls  with  at  most  /  switches,  which  converges  uni¬ 
formly.  This  approach  was  used  to  obtain  a  maximum  solution  of  certain  (QVI)  in 
Menaldi  [10-11]  without  nondegeneracy  assumptions.  In  Blankenship  -  Menaldi  [12], 

related  problems  were  treated  involving  the  application  of  (QVI)  to  power  generation 

biases  with  white  noise  models  are  treated  in  [5]  and  [6],  while  control  problems  for  diffusion  processes 
with  jumps  are  treated  in  Bensoussan  [7).  See  also  [8]  for  an  introduction  to  the  subject. 


systems  with  scheduling  delays.  See  also  [13] [14]  for  a  survey  of  viscosity  methods  for 
the  control  of  diffusions. 

The  optimal  stochastic  control  of  linear  regulator  systems  with  Poisson  noise  distur¬ 
bances  is  considered  in  [15];  stochastic  stability  properties  of  linear  systems  with  multi¬ 
plicative  Poisson  noises  are  derived  in  [16].  See  also  [17]. 

1.1  Problem  Statement. 

Let  (O ,F,P)  be  a  probability  space  and  {Ft ,  t  >  0}  a  non-decreasing,  right- 
continuous  family  of  completed  sub  cr-flelds  of  F  such  that  Ft  |  F ^  =  F ,  t  >  0.  Con¬ 
sider  the  general  nonlinear  dynamical  system 

I  dyx(t)  =  g(yx(t),a(t))dt  +  h(yx(t),a(t))dN a[t){t) 

\  yx  (o)  =  *  (1,1) 

where  Af-(l),  i=l,...,mf  are  independent  Poisson  processes  with  intensities  X,- , 
a{t)  is  a  right  continuous,  piecewise  constant  random  function  with  finite 
range  ,  and  is  measurable  with  respect  to  Ft ,  t  >  0.  Actually,  a  is  an  admissible 

control  consisting  of  random  switching  times  and  random  switching  decisions  d{  such 
that  di  are  adapted  to  {Ft  }  and  d,-  are  F$  -  measurable  so  that 

0  =  90  <  91  <  •  •  •  <  (9^!  <  <  ei+1,  6i  ->  +  oo  a.s. 

di  E  {l ,  .  ■  ■  ,  m  },  d{  di_ 1  if  <9;  <  oo  (1.2) 

And  so 

a(t)  =  di  if  6i  <  t  <  9i+1,  i  >  0 
is  indeed  Ft  -  measurable. 

Let  the  set  of  all  admissible  controls  with  initial  setting  d  be 


2 


Ad  t  {a  |  a  =  {0j ,  di  }  satis f  ies  the  above  properties 


(1.3) 


with  initial  setting  d0  =  d  }. 

We  take  the  performance  index  to  be 

OO 

0°  R{. 

Jx\a)  t  ExJ{jf(yx(t),a(t))e-^dt  +  *} 

0  i  =1 

6, 

oo  . 

=  /  /  (vAn.di-Je-Vdt  +  kid^df  )e~pe'}}  (1.4) 

i  =1  e,_  j 

where  (3  >  0  is  a  discount  factor  and  k(d  ,d)  is  the  cost  of  switching2  from  d  to  d  such 
that 

k  (d , d)  >  0  if  d  yA  d\  k(d  ,d)  —  0  (1.5) 

k  {d  ,d)  <  k  (d  ,d)  +  k  (d  ,d)  if  d  ^  d  d  . 

Without  loss  of  generality,  we  can  define  k0  =  min  {  k  (d  ,d),  d  =£  d  }.  We  assume  / 
>  0,  g  and  h  are  bounded  and  Lipschitz  continuous 

\  q(x  ,d)\  <  | \q  1 1  <  oo 

\  q  (x  ,d)  -  q  {x  ,d)  \  <  L  \  x  -x  \  (1.6) 

with  q  —  f  ,  g  and  h  ,  for  all  x ,  x  £  IR,! ,  d  £  1  . 

Under  these  assumptions,  (1.1)  has  a  unique  solution.  Defining  the  value  function 

ud(x)  ~  inf  Jd{a),  x  £  1R" ,  d  £  {l,  .  .  .  ,  m}  q  7\ 

we  want  to  design  an  optimal  control  a*  such  that 

ud(x)  =  Jxd(a* )  =  inf  Jd(a).  n.s) 

a€Ad 


2The  case  when  the  switching  costs  can  be  zero  is  treated  in  section  5. 
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Remark.  iVa(()(f)  is  an  inhomogeneous  Poisson  process  with  intensity  function 

^a(ty 

1.2  Summary  of  Results. 

In  section  2  we  show  that  the  optimal  value  function  ud(x)  in  (1.6)  maybe  defined 
as  the  limit  of  the  value  functions  uf{x)  of  systems  with  a  finite  number  /  of  switches  as 
/  — >  oo  (Theorem  2.3).  We  show  that  the  convergence  is  uniform  (Theorem  2.5);  and  we 
derive  two  representations  of  ud(x)  as  the  optimal  value  function  (Theorems  2.6  and 
2.7).  We  describe  the  associated  optimal  (control)  switching  policy  (Theorem  2.8),  and  we 
use  it  to  obtain  an  additional  estimate  on  the  convergence  of  uf  to  ud . 

In  section  3  we  derive  the  QVI  which  must  be  satisfied  by  the  optimal  value  func¬ 
tion  (equation  (3.3)).  We  show  that  the  optimal  value  function  is  a  viscosity  solution  of 
the  QVI  (Theorem  3.1).  Then  we  show  that  the  solution  is  unique. 

In  section  4  we  prove  that  the  QVI  has  a  viscosity  solution  by  constructing  a 
sequence  of  solutions  to  a  penalized  system  (equation  (4.2))  and  proving  that  hese  solu¬ 
tions  are  uniformly  bounded  and  uniformly  HOlder  continuous  (Theorem  4.4).  We  show 
that  the  limit  of  the  sequence  of  solutions  to  the  penalized  system  is  a  viscosity  solution 
of  the  QVI. 

In  section  5  we  consider  the  case  when  the  switching  costs  vanish  ( k(d,d )  =  0  for 
d  7^  d  in  (1.4)).  In  this  case  the  optimal  value  function  u  is  independent  of  the  initial 
control  configuration  d  (since  we  can  switch  for  “free”  at  any  time),  and  it  (formally) 
satisfies  a  Hamiliton  -  Jacobi  -  Bellman  equation  which  is  fully  nonlinear  in  \ju  .  The 
method  of  viscosity  solutions  is  required  to  treat  this  case.  We  show  that  the  optimal 
value  function  corresponding  to  non-zero  switching  costs  will  converge  to  u  as  the 
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switching  costs  tend  to  zero,  and  that  u  is  the  unique  viscosity  solution  of  the  Hamiliton 


-  Jacobi  -  Bellman  equation.  The  result  is  analogous  to  those  in  Capuzzo  Dolcetta  - 
Evans  [4].  Thus,  the  method  of  viscosity  solutions  provides  a  complete  framework  for  the 
treatment  of  the  optimal  control  problem  (1.1)  -  (1-8)  over  the  full  range  of  parameter 
values  and  operating  regimes. 

2.  Dynamic  Programming  and  Some  Preliminary  Results. 

Before  using  dynamic  programming  to  investigate  the  properties  of  the  value  func¬ 
tion  ud(x),  we  need  some  preliminary  results. 

Lemma  2.1.  For  any  stopping  time  t  which  is  adapted  to  {Ft  }  and  any  measur¬ 
able  bounded  function  q,  we  have 

E  { q(yx(t+r ))  |  Ft]  =  EyAT)q(yyAT)(t)).  (2.1) 


Proof.  Since 


T+t  T+t 

yx(t  +T)  =  Vx(T)  +  J  g(yx(s),a(s))ds  +  f  h(yx(s),a(s))dNa($)(s) 

t  r 

t  t 

=  Vx(T)  +  jg{yx(s+T),a(s))ds  +  J  h{yx{s +f),a(s))dN  a{s+Tp) 

0  0 

(2.2) 

where 

a(s  )  —  a(s  +r) 

N  i(s)  =  Ni(s  +r)  -  N{  (r). 

We  claim  JV;( s)  is  a  Poisson  process  which  is  independent  of  r.  Since  t  is  adapted  to  Ft 
and 


F T  =  {  B  e  F  [  B  f^{r<  t}  E  Ft  }, 
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we  know  r  is  immeasurable.  Thus 


P  \Ni  (s  )=k]  =  P  [Nt  (s  +r)~Ni  ( T)=k  ] 

=  E  {  P  [N{  (s  +t)—N{  (r)—k  \  FT}}. 

Since  a  right  continuous  Poisson  process  is  a  Feller  process,3  then  is  a  strong  Markov 
process.  Thus,  for  £  >  s  , 

P[Ni(t)  -  Ni(s)=k]  =  P[A;-(f+rm(s+H] 

=  E  {P  [Nt  (£  +T)~Ni  (s  +r)=k  |  Fs  +  r]} 

=  E  {P  [Ni  (t  +T)-Ni  (s  +r)=k  |  Nt  (s  +r),  r]} 

__  t\(£-s  )}k  - x, (< -» ) 

k !  6 

=  p  [Ni(t  +t)-N{  (s  +r)=k  j  ^  (s  +r),  r]  (2.3) 

Moreover,  if  0  <t0<t1<-  -<tl  <  oo,  and 

Niitj)  -  Ni(tj_1)  =  Niitj  +r )  -  Niitj^+r), 
then  by  the  strong  Markov  property  and  (2.3) 

P  {N i(t0)=k0,  N iitJ-N i(t0)=k  v  .  .  .  ,  Ni(t[  )-N i  (t[_1)=ki  } 

=  E  {In,  (t0  +  t)~n,  ( T)=k0  In,  (t ,  +  t)-n,(i0  +  r)=* ,  '  '  '  In,  (t,  +  r)-w,  (<, _j  +  r)=k , } 

=  E  ^f+i  times  E  ^N'  {t°  +  r)‘/V.(f)=<:o  ‘  '  '  (t,  +  r)-N,(t M  +  r)=k,  I  Ft M  +  r  }  '  ’  '  I  Ft0  +  r}} 

^  E^EStimes  E  {lNA‘o  +  T)~N,(T)=k0  '  ’  '  +  t)-N,  (t,_2  +  r^k,^ 

'  P  {In,V,  +T)~N,{t,_l+T)=kl  I  +  T)>T  }  |  Ft,_ 2  +  t}  '  '  '  |  ^(o  +  r}} 

^  ^E Stimes  ^  {^N,  (lo  +  t)-N,  M=*0  (h-i  +  r)-^V,  (h-2  +  r)=^/-i  I  ^(-2  +  I  ^0  +  r}} 

• }  (2.4) 

3See  E  B.  Dynkin,  Markov  Processes,  Vol.  1,  Spnnger-Verlag,  Berlin,  1965  (section  2.18,  pp.  69). 
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where  I  denotes  the  indicator  function.  By  induction,  (2.4)  is  equal  to 


P{N{(t0)=k0}  ■  ■  ■P{ffi(t,)-Ni(tl_1)=kl}, 

so  that  N i  has  independent  increments.  Thus,  N ^  is  the  Poisson  process  with  intensity 
Xj  .  Since  y  y  (r)(t)  =  «4(2+r)  is  a  unique  solution  of  (2.2),  the  statistics  of  y  x  (t)  and 

yx  (t)  are  the  same  if  a  =  a.  From  (2.2),  we  know  that 

E  {q  (yx  ( t  +r))  I  Ft}  =  Eyt{T)  q  (yx  ( t  +r)) 

=  Eyt  (r)  q  ( y  yx(T)(t )) 

=  EVA*)  ?(%(r)(0)- 

QED 


Lemma  2.2.  For  each  d  £  {l  m}  and  x  £  1R”  , 

(i )  ud  (x)  <  min  {ud(x)  +  k(d,d)}  (2.5) 

d  y^d 

(ii)  For  any  stopping  time  0  >  0, 

0 

ud(x)  <  E  {J  f  (yx(s),d)e^s  ds  +  ud(yx(9))e^e}  (2.6) 

0 

Proof,  (i)  Let  d,  d  £  {l,...,m}  be  such  that  d=^=d  .  Set  a  =  {#,- ,  d±  }  £  Ad . 
Define  a  =  {6ir  d^-Zo  by  60  =  0,  d  0  =  d ,  0£  =  0i_1,  d  t-  =  <4_x  for  i  >  1.  Then 
#.  =  $i_1  is  adapted  to  {Ft  }  and  d{  =  d{^  is  F9<  -  measurable,  so  that  a  £  A  d . 

Thus, 

ud{x)  <  Jd(a) 

=  E  {  £  L  f  (Vx(s)’d  i-i)e  ds  +  k  (d  i_lrd  8-  )e  ^  1 } 

i=i  ' 9-1 
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Since  a 

(ii) 

by 

Then  0{ 
a  e  A 


~  oo  _  . 

—  E  {  k  (d  ,d)  +  E  J  f  (yxis),di-i)e  ~  ^  ds  +  k  (di__1,di)e  ~  P '} 

i  =1 

=  k  (d  ,d)  +  j}{a). 


6  A  d  and  d  are  arbitrary,  we  know  that 

u  d  (x  )  <  min  {  k(d  ,d)  +  ud(x)}. 

~  d^d. 

Let  a  —  { 0i ,  di  }  G  A  d  and  fix  a  stopping  time  0  >  0.  Define  a  =  {0j ,  d  i  } 


90  =  0,  d  0  —  d  ■, 

L  (9,  c?  ,•  =  ,  *  >  l . 

=  +0  is  adapted  to  {Ft  }  and  d  ,•  =  di  is  -  measurable,  so  that 

^  .  Thus,  from  Lemma  2.1, 


u^O)  <  //(a) 


* 


0,  +0 

E  /  f  (yx(s)’di-  i)e~Psds  +  k(di_1,di)e  ~^6,  +$) 

i  =1  0,_!  +  9 


=  £ 


9 

J  f  (yx(s),d)e  -Vs  ds  +  E 
0 


t 

00 

E  /  /  (y*(5+0).<*»-i)c  * 

!  =  1  0,  _i 


-/-  A’  (  d.^  j ,  dL )  c 


Fe 


=  E 


9 

f  f  (yx(s),d)e  ~  0S  ds  +  EVi{ft) 
0 


00 

E  /  /  (y,jjo)(s),di  i)e  rfs 

i  =1 
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+  k(di_1,di)e-pSt  j  e  ^  j 


0 

f  f  (yx(s)>d)e  ~0S  ds  +  Jy[  {g]  (a)  e~m 
0 

Since  a  £  A  d  is  arbitrary,  we  have 

0 

ud(x)<E  -  /  /  (yx(s),d)e  ~^s  ds  +  ud  (yx(0))  e  ~  pe 
o 

QED 

Notation.  For  x  £  IRn  ,  d  £  1  , 

Md[u}(  x)  =  min{u^(a:)  +  k(d,d)}.  r2.7) 

Now,  we  want  to  use  the  dynamic  programming  principle  to  show  there  exists  a 
convergent  sequence  {ud}  of  optimal  solutions  of  the  problem  with  respect  to  controls 
which  have  at  most  l  switches. 

For  each  x  £  IRn  ,  d  £  1  ,  let 

00 

u$(x)  =  /  /  (yx{s),d)e  ds.  (2.8) 

0 

Notation.  If  u,v  £  C  (IR”  )m ,  then  we  say  u  >  v  if  u d  >  v d ,  V 
d  —  1,  .  .  .  ,  m. 

Define  an  operator  Frf  :C  (IR”  )m  — +  C  (IR'* )  by 
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(2.9) 


0 

J  f  (yx{s),d)e  ds  +  e  ~&eMd  [u  }(yx  (0)) 
o 

Here  we  understand  the  inflmum  is  taken  for  all  stopping  times  9  >  0  adapted  to  {Ft  }. 
If  u  >  v  ,  then  for  each  e  >  0,  there  exists  a  stopping  time  9e  >  0  and  de  Fs  -  measur¬ 
able  such  that 


r  du(x)  =  inf  E  { 

0>O 


Tdu(x)  >  E 


f  f  {yx{s),d)e~pt  ds  +  e  ~^{u  d\yx  (9e))  +  k(d,de)} 


>  E 


f  f  (yx(s),d)e  Vs  ds  +  e  P°c[vi({yx(de))  +  k(d,d 


>rdv(x)  -  e. 


Let  e  |  0,  we  have  Tdu  >  v .  Let  0  <  rj  <  1,  then 

rrf  [(l-»/)u  +  rjv] 

0 

J  f  iyx(s),d)e  ~0S  ds  +  e  ~^Md  [(l~v)u +r}v](yx{6)) 
0 


e 

J  /  (Z/zO  )>d)e  ~P*  ds  +  -vWd  \u}{yx{9))  +  r)Md  [v  }{yx  (0))}  • 

0 

>  (1  -  Tj)rdu{x)  +  r}Tdv{x). 

Thus,  is  a  non-decreasing,  concave  function. 


>  inf  E 

~  0>O 


=  inf  E 

0>O 
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Suppose  we  are  given  Ui_x. 


We  can  define 


0)  =  rdui- i(x)- 

Since  uf(x)  =  r^u0(a:)  <  ud(x),  then  by  the  non-decreasing  property  of 
u  g  (x  )  =  T  fu  0(x  )  <  u  0{x  ),  and  so 

0  <  ut\x)  <  u,li(ar)  <  •  ■  •  <  ud(x)  <  -W/  H  . 

Thus,  ud(x )  converges.  We  can  define 

«£>(*)  =  lim  uf{x). 

I  — >00 


Theorem  2.3. 


ud(x )  =  inf  {  Jd(at)  \  otl  E  A  d  has  at  most  l  switches  } 


and  thus 


u^(x)  =  ud  (x)  =  inf  Jf(a). 


a  €  A  ‘ 


Proof.  Clearly,  (2.13)  is  satisfied  for  1=0.  Suppose  (2.13)  is  true  for  /- 
switching  policy  a*_x  =  {0-~1,d,dl~1'd}  E  Ad ,  \f  d  =  1,  m  .  Then  let 

9  =  inf  { stopping  time  6  >  0  |  ud_x  ( yx(0 ))  ==  Md  [ul_1](yx(9))  a.s.  } 

and 

d  =  any  F-g  -  measurable  randam  variable  d  d  such  that 

Md  [ul_1]{yx  (0))  =  ud  L  (yx  (0))  +  k  (d  ,d  )  a.s. 

Now,  we  can  define  af  —  {9-’d,  d- ,d}  by 


9‘J=0,  d‘oa  =  d 


l,i 


(2.10) 
,  we  have 

(2.11) 

(2.12) 

(2.13) 

(2.14) 

1  and  the 

(2.15) 

(2.16) 
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+  0,  1=1,  .  .  .  ,  l 


(2.17) 


c\  l  ,d  _  n  1—1,3 

“i  —  “i  -l 

d[’d  =d,  dlA  -  dil\  i=  2,  ,  l. 

Then  dl’d  =  9  and  by  induction  9-  ’d  9-~l'd  +  9,  2  <  i  <  /,  are  adapted  to 
while  d[‘d  =  d  and  d-  ,d  =  d-ll’3 ,  2  <  i  <  /,  are  -  measurable. 

a*  E  A  d  .  By  the  principle  of  dynamic  programming, 

ud{x)  £  //(«/),  al  has  at  most  /  switches. 

On  the  other  hand,  by  induction  and  Lemma  2.1, 


ud(x )  =  E 


f  f  {yx{s),d)e  0S  ds  +  k(d,d)e  +  u3_l{yx{0))e 
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=  E 


f  f  (yx(s)'d)e  /3s  ds  +  k{d,d)e 


+  EaIW.d 


E  / -/ 

i=l  0  l~}'d 


ds 


+  k{dl:l’3 ,dl~x-3)e 


£ 


/  /  {yx(s)>d)e  0S  ds  +  k(d,d)e 


l-i 

E 


+  o 


f-  f  {yx(s),dl-i3)e  1)8  ds 


Hd£t3  ,dl-1-3)e~*9l~l'i  +  J) 


,1-ij  +  e 


{Ft} 

Thus, 
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E  E  /  /  (y,(a),^)e 


ds  +  k(d‘i,d‘J)e 


i  =  1  $IJ 


(2.18) 


//(«/*)•  . 


Thus  (2.13)  and  (2.14)  follow. 


QED 


Lemma  2.4.  For  each  0  <  7  <  min  J  1, 


«A»)  -  «/(*)  I  <CAx  -  X  |7 


(2.19) 


/or  all  1  <  l  <00  and  a: ,  x  €  IR"  with 


C  =  _ I.U..J.J _ L _ 

7  (3  -  (i+Xmax) 


(2.20) 


where 


^max  max  {Xj,  ■  ■  ■  ,  X m  }. 

If  f3  >  L(1  +  Xmax),  £/ien  7  can  6e  taken  to  be  l. 

Proof.  Without  loss  of  generality,  we  can  assume  uf{ x )  >  n/(x ).  Let 
—  {^i »  d  i  }  £  A d  be  the  optimal  policies  with  at  most  /  switches  at  state  x  . 


Then, 


Ui\x)  -  a  fix  )  |  <  u/(a: )  -  ufix)  <  Jxd(at)  -  JUott) 


<E  J  \  f  iyx{t),al(t))  -  f  iyi  {t),al{t))\  e  ,n  dt  (2.21) 
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t 

yi(t)  -  Vx(t)  =  x  ~  x  +  J  Id  (Vx  («)-«/(>))  -  g  (yj  (s),al(s  ))}ds 

0 

t 

+  f  [h(yl(s),al(s))  -  h(yi  (s),al(s))}dNaiis](s), 
o 

then  taking  the  expectation,  we  have 

t 

E  |  yi  ( t  )-yj  (<)|  <  |  x-x  |  +  L  (l+Xmax)  J  E  \  ylx{s)-yj  (s  )  |  ds. 

0 

By  Gron wall’s  inequality 

E  |  y‘(thyi  (01  <  \x-x  \  eL{1+^]t, 

so  that 

E  |  /  (lli(t),a,(t))  -  f  (yi  {t),at{t))\ 

<  11/  \n\E  \f(yi(t),a,(t))  -  f(yi(t)~at(t))\r  (2.22) 

<  ||/  ip-u7!  x-x  |  v  oL(1+Xm“)' . 

Thus,  from  (2.21), 

oo 

\ut\x)  -  ut\x)  |  <  ||/  W^L^x-x  |7  /  dt 

0 

_  11/  I r-i-1  ,, 

P~1  L  (l  +  ^max) 

=  (77  |  X-X  |  7. 

For  l  =  oo, 

\uio(x)  -  ui0(x)  |  <  |  (x )  -  ut\x )  | 

+  [  w/(a: )  -  |  +  |  m/(x  )  -  (x)  \ 

<  Cn  I  X  -  X  I  7  +  \uioa(x)  -  ut\x )  I 
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+  |  utd{x)  -  u^,(x) 

As  /  — ►  oo,  the  last  two  terms  tend  to  zero.  We  have  (2.19). 


QED 


Remark.  Since  A/,-  has  independent  increments,  then  Fs  is  independent  of  any 
sub  u-fleld  generated  by  {/V,-  ( t )— TV,-  (s ),  s  <  t ,  i  =  1,  .  .  .  ,  m  },  so  that  for  t  >  s, 

E  [  |  y‘(t  )~yt  (0|  I  Fs )  <  |  y‘(s  )  -  yj(s)\  ei(1+W«-*) 

Thus, 

I  u  i  (Vx(s  ))  -  ud(yl  («))  I  <  C1  I  yl(s)  -  yi  (s)  |  7  a.s. 

Remark.  If  k0  >  ■  1 1  ^  II-  ,  then  «0(i)  is  the  optimal  solution,  i.e.,  no  switching 

P 

occurs. 


We  can  obtain  the  following  estimate  by  the  method  in  [10] [11], 


Theorem  2.5.  If  0  <  k0  <  I  ^  *-0  then 


W  -  ut,  ||  <  ||«orf 


ii  d  ii 

\  Pko  \ 

1  iuo  II 

l 

f— H 

(2.23) 


Thus,  U[  |  «oo  uniformly. 


Proof.  Using  the  fact  that  °  +  ™ ■  >  —  if  a  >  0  and  w  <  z ,  we  have 

a  +  z  z 
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I\j(0)(ar)  =  inf  E 
0>o 


8 

f  f  {yx{s),d)e~0s  ds  +  epeMdmyxm 
o 


>  inf  E 
0>o 


0 

f  f  (Vx  (s  )>d)e  ~Ps  ds  +  k0e 
o 


>  inf 

0>O 


0 

E  f  f  (yx(s),d)e  ds  +  koe~0e 
0 


00 

E  J  f  ( yx(s),d)e  ~0S  ds 
0 


«o  (*) 


>  inf 

0>O 


E  k0e 


00 


E  f  f  ( yx(s),d)e  0S  ds 
0 


ud{x) 


>  inf 
0>o 


E  k0e 


E 


11L 


0 


« oOO 


> 


Uq(x) 


=  i louo(x )• 


Thus,  if  -  p2v  <  u  -  v  <  p  iU  ,  0  <  p  v  p  2  <  1,  and  0  <  u ,  v  <  u0,  then 
Tdv{x)  >  rrf  (0)(a: )  >  q0ud0{x)  >  q0ud(x )  >  q0Tdu{x) 


so  that  by  the  property  of  , 


(2.25) 
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Vdv{x)  >  rrf[(l-pi)«](a:) 


Thus, 


=  [(i-pi)m  +  PiO}(x) 

>  (l-P  i)rrf  u  (x  )  +  ptrd  (0)(ar). 


Tdu{x)  -  rdv(x)  <  p  X\T  du{x)  -  rd(0)(a:)] 

<  Pi(i-qo)rdu(x). 


On  the  other  hand, 

rrfu(aO  >  rrf  [(i-p2)u  j(aO 

=  [(1-P  2)^  +  P2°]0) 

>  (1  -P2)rdV(x)  +  p2rd(0)(x), 


so  that 

rdu(x)  -  r dv(x)  > -p2[rdv(x)  -  rrf (o)(ar )] 

>  -P2(1-?o)rrf  <>(*)• 


Hence,  by  induction, 

-P2(1-(lo)‘rdv(x)  <rju(x)  -  F  J  v  (x  )  <  Pl(l-q0)‘rju(x),  V  l 
Note  that 

uq(x)  >  rdu0{ x)  >  >  r du0(x )  >  rjw(x) 

and  with  p  t  =  p  2  —  1,  i  >  / , 

\W  -  uf  II  =  ||rj«o  -  rj«;_,  II  <  (l-9o)'  I  l«o  11- 

Let  i  —*  00  and  the  result  follows. 


>  1.  (2.26) 


QED 
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Theorem  2.6. 


8 

UL(X)  =  jpf  {E  f  f  (yx(s  ),d)e  -0s  ds  +  Md  [Uoo\(yx  (d))e  ~ }.  (2.27) 

8>0  0 

Proof.  Since 

UL  (x)  —  hm  ud{x) 
l  — >00 

8 

=  Um  jnfE  {/  /  ( yx(s),d)e  ~^s  ds  +  e  ~PeMd  [ul_l}{yx  (0))} 

and  <  U[  <  <  u0,  then 

Md\u oo]  <  Md[Ul }  <  Md  [«/_x] 

so  that 

o 

uLi*)  >  inf  E  {$  f  {yx{s),d)e-f)s  ds  +  e  "  0eMd  \u  «,](&  (9))}. 

o 

On  the  other  hand,  from  Theorem  2.5,  ut  {  uniformly.  Thus,  for  each  small  e  >  0 
and  stopping  time  6  >  0,  there  exists  /  sufficiently  large  so  that 

Md[u00){yx{d))  >  Md[nl_l\{yxm  ~  « 

and  so 

6 

E  {  /  /  (yx{s),d)e -Psds  +  e  ~ 0eMd  \u  J(y#(tf))} 

0 

9 

>E  {/  /  (yx(s),d)e-^ds  +  e  ~^Md  [u^y,  (0))}  -  e 

0 

>  u/(a: )  -  e. 

Letting  /  — *•  oo,  taking  infimum  for  all  stopping  time  0  >  0  and  dropping  e  |  0,  we 
know 
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inf  E  { J  /  (yx(s),d)e 
e>o 


+  e  peMd  [u^iy^d))}  >  u^ix). 


Thus,  we  have  the  desired  result  (2.27). 


QED 


Theorem  2.7.  If  zj  x  0  such  that  ud  (x0)  <  Md  [u}(x0),  then  0X  >  0  a.s.  and 


(x0 )  =  E  {/  f(yXo(s),d)e  ps  ds  +  ud(yXo(0))e 


(2.28) 


for  all  0  <  6  <  6V 

Proof.  Suppose  not,  P{0X  —  0)  =  a  >0.  Let  fi,  =  {w  £  fl  |  6X (ui)  =  o}.  If 

a  =  min  {ud(x0)  +  k(d,d)}  -  ud(x0)  >  0, 
d 

then  from  (2.27),  0X=£  0  a.s.,  so  that  a  <  1.  Since  yxft)  is  right  continuous,  and  /  is 
bounded,  then  for  t  small  enough, 


ud(yXo(t  ,qj))  e  <  ud{ x0)  +  — 

o 


(2.29) 


for  all  w  £  fi2  with  P  (fl2)  >  1  -  —  and 


(2.30) 


Thus,  P(f21p|f23)  >  —■  •  Let  O3  =  n  \  (niP)n2).  Then,  from  (2.27),  (2.29)  and  (2.30), 
2 


we  have 


!(z0)>£  {//(y,0(s  ),d)e-f>sds  +  u  d  (y,^))  e  ^8l}  /c 
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+  E  {  /  /  iyx0{s),d)e  08  ds  +ud{yXo(t))e  0t  } 


9-y{oS),  OJ  £  f7g 


t }  uj  £  • 


Then  it  is  easy  to  check  that  6  is  adapted  to  {Ft  }  and 


'(x0)>E  {/  f  (yXo(s),d)e  08  ds  +  ud(yXo(d))e 


which  contradicts  (2.6).  Hence,  a  =0.  By  (2.27)  and  Lemma  2.1,  we  have  for  any 

0  <0  <0V 


'( *o)  =  E  {/ /  (yXo(s  )>d  )e  ~0S  ds  +  Md  [u  ](y,0(^))e 


=  £■{// (&„(«  )’^)e  ***  +  #[J7  (yXo(s)>d)e  Ps  ds+Md[u}(yXo{91))\Fg\} 


E  {  /  /  {yXo{s),d)e  08  ds  +  (g)  f  f  (yyi  {S)(s  ),d)e  05  ds 

0  o  0 


Md  [« }(yyioW(0i~0)e~m-$)' 


>E  {/  f  (yXo(s),d)e-0s  ds  +  u  d  (yXQ(0))  e  ~  09}. 


By  (2.6),  we  have  (2.28). 


QED 
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Now,  suppose  we  have  a  HOlder  continuous  function  u d  satisfying  (1.6).  We  can 
define  an  optimal  policy  a*  =  {0t ,  d{}  E  Ad  as  follows. 

0o  =  0,  d0  =  d , 

If  we  are  given  0{ _Xj  d^v  then  set 

6i  t  inf  {stopping  time  0  >  |  u  d‘ \yx  (<?))  =  Md'-l\u  ]  (yx  (<?))  a.s.  } 

(2.31) 

If  Oj  <  oo,  set 

di  =  any  F -  measurable  random  variable  d  E  {l,  .  .  .  ,  m),  d  d{_x 

such  that  Md'-l[u){yx{di))  =  ud{yx{0i))  +  k{d  ~d)  a.s.  (2.32) 

and 

yx  (t )  controlled  by  decision  when  9t_ x  <  t  <9^ 

Theorem  2.8.  The  control  policy  a*  defined  by  (2.31)  and  (2.32)  is  optimal,  i.e., 


ud{x)  =  Jx\a*)  = 

min 

a  e  AJ 

.  In  addition,  9i  — ►  oo  a.s.  as  i  — ►  oo. 

Proof.  Set  2/j_ 

1 

I! 

.  By  Theorem  2.6  and  Lemma  2.2, 

1)  =  Ey, 

(  8,  -8,. 

f 

0 

-1 

fiVyJsMt-Je-^ds  +  «<U(yi)e’W~*-l) 

=  E  < 

st 

f  f  ( yx(s 

).di-i)e  ^(S~6'-l)ds  +  u^y^e-**'-9'-1'  F ^ 

Since 
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u  d'\yi )  =  Md'~l[u  }(Vi  )  =  ud\yi)  +  k  {di^di )  a.s. 


we  have  by  Lemma  2.1, 


~  ^^1-1  ^t-i/  \ 

e  «  (Vi~  i) 


f  f  (yx(s),di_1)e  Ps  ds  +  k{di^1,di) 


«  *  +  «  ’(y,-)  c 


Summing  up  all  i  until  some  dt  —  oo  and  taking  expectation,  we  have 


u  (x)  =  E 


£  /  f  (yx(s),di_1)e  0S  ds  +  k(di_1,di)t 

i  >  1  6,  _j 


=  4V). 


As  from  (1.6),  we  have  proved  a*  is  optimal.  We  claim  0,-  -*  oo  a.s.  as  i  -> 
pose  not,  then  =|  T  >  0  such  that  0,  <  T  for  all  i  with  positive  probability 
from  (2.11)  and  (2.33), 


11/  11 
P 


>  T.E  kidi.Me-09'  >  ^k0e~^8 

i  >  1  »'  >  1 


which  is  unbounded,  a  contradiction. 


Corollary  2.9.  We  have  the  additional  estimate 


\W 


11/  II2 

P2k0(l  + 1) 


(2.33) 


oo.  Sup- 
8.  Then 


QED 

(2.34) 


22 


Proof.  From  (2.31)  and  (2.32),  we  obtain  an  optimal  policy 
a*  =  {#j ,  di }  (E  yl .  Let  ==  {0/,  <f/}  (E  Ad  be  the  first  /  switches  such  that 
0/  =  0,- ,  of/  =  d-i  for  i  <  /  and  0 ■  =  oo  for  i  >  l .  Denote  by  yj(t )  and  yx  ( t )  the  tra- 
jectories  corresponding  to  the  controls  and  a  ,  respectively.  Then  yx  ( t )  =  yx(t)  for 
0  <  t  <  9l+1.  Hence, 

0  <  U[d(x )  -  «£>(*)  <  //(«()  ~  //(«*) 

OO 

<£  /  fivtisUrie-t'ds 

-4-1 


<  11/  11  £  e 

“  /? 


(2.35) 


Since  k(di,dj )  >  A0,  i  7^  j  a.s.,  and  0,-  <  0/+1,  i  <1+1  a.s.,  we  have 


-W+i 


< 


~  *0(/+ 1)  i=i 


1+ 1 

S  k(di_1,di)e 


a.s. 


(2.36) 


Since 

u£>  (*)  =  Jd(a*)  =  E  {  £  /  f  (yx(s)'di- i)e~psds  +  k(di_1,di)e~0S'} 

i  =1 


we  have 


E  £  k(di_1,di)e  ~PB'  <  -LL^-U-  ,  (2.37) 


so  that  from  (2.34)  -  (2.36), 


0  <  1^(2: )  - 


«£>  OO  < 


11/  ll2.,. 

P2k0(l+1) 
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for  all  x  (E  IR.”  .  Thus  (3.18)  follows. 


QED 


3.  Viscosity  Solutions  of  the  Quasi-Variational  Inequality  (QVI). 


We  want  to  derive  necessary  and  sufficient  conditions  for  the  optimal  solution 
ud(x),  x£  JRn ,  d  E  Assume  for  the  moment  that  the  value  functions 

u  1,  .  .  .  ,  u  m  belong  to  6(1(IR”).  Then  by  the  necessary  condition  in  Lemma  2.2,  we 
have 


E 


ud(x)  -  ud  (yx(t)) 


+ 


-Pt 


<  E  \  -j  f  f  ( yx(s)>d)e  ^  da 


J  ud{yx(t))  j 


(3.1) 


and  so,  we  obtain  a  differential  form  as  t  {  0, 


-  g  (x  ,d  )-\/u  d  (x  )  -  \d  [u  d  (x  +h  (x  ,d ))  -  u  d  (x )]  <  /  (x  ,d  )  -  f3ud  (x)  (3.2) 

for  all  x  E  IRn  and  d  E  1  Combining  (2.5),  (2.28)  and  (3.2),  we  obtain  a  quasi- 

variational  inequality  (QVI) 


max 


{/3ud  -gd-vud  ~\d[ud{-+hd)-ud}~  f  d ,  ud  -Md[u]  }  =  0  (3.3) 


on  IR"  ,  where 

gd{  )  t  g(-,  d),  hd(-)  t  h{-,d).  (3.4) 

Note  that  (3.3)  is  a  fully  nonlinear  first  order  partial  differential  equation  which  does  not 
admit  a  differentiable  solution  in  general.  But,  we  can  treat  it  using  the  method  of 
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viscosity  solutions,  which  was  introduced  by  M.  G.  Crandall  and  P.  L.  Lions  [1],  and 
which  was  used  for  deterministic  switching  problems  by  I.  Capuzzo  Dolcetta  and  L.  C. 
Evans  [4]. 

We  denote  by  BUC  (K“  )m  ,  the  space  of  bounded,  uniformly  continuous  ]Rm  -valued 
functions  on  IR.”  . 

Definition.  A  function  u  =  (ud ,  ...  ,  um)  £  C(IRn  )m  is  said  to  be  a 
viscosity  solution  of  the  (QVI)  if  for  each  d  £  {l,...,m}  and  each  £  C'1(IR'i )  such  that 

(i)  if  u  d  -  f  attains  a  local  maximum  at  x0  £  IRn  ,  then 

max  {  /3u  d  (x  0)  -  g  d  (x0)- vf(x0)  ~  ^d  [«  d  (x0+h  d  (x0))  -  ud{x  0)]  -  fd(x0), 

ud{x0)  -  Md  [u}(x0)  }  <0  (3.5) 


and 

(ii)  if  ud  -  <t>  attains  a  local  minimum  at  z0  £  IRn  ,  then 

max  {  fdud  (z0)  -  gd  {z 0)-\/ 0)  -  \d  [ud  {z0+hd  (z0))  -  ud(z0 )]  -  fd{z0), 

ud(z0)  -  Md  u](z0)  }  >  0.  (3.6) 


Theorem  3.1.  Under  the  previous  assumptions,  the  value  function  u  = 
{u 1 ,  .  .  .  ,  u  m  )  with 


ud(x)  t  inf  Jd(a) 

a  £  Ad 


is  a  viscosity  solution  of  the  (QVI)  (3.3). 

Proof.  By  Lemma  2.4,  u  £  BUC(JR,n  )m .  Now,  let  <f>  £  C^IR").  If  u d  -  f 
attains  a  local  maximum  at  x0  £  IR”  for  some  d ,  then 
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U  O o)  -  ^Oo)  >ud{x)  -  4>{x) 


(3.7) 


for  all  x  in  some  ball  B{x0,e),  with  e  sufficiently  small.  By  Lemma  2.2  (ii),  we  have 


E 


ud{x0)  -  ud{y  (0) 


<i?  {  1/  f(yXo(s),d)e^sds 
f  0 


— 1  )  «'(y,0(O) 


(3.8) 


Since  g  is  bounded,  Lipschitz  continuous  and  yx  is  right  continuous,  then  for  sufficiently 
small  t  >  0 


■i(t)  t  x0  +  f  g  (yXg(s  ),d  )ds 


(3.0) 


belongs  to  some  ball  D(x0,e)  a.s.,  so  that  by  the  mean  value  theorem, 


ud(x0)  -  u  d(x1(t))  \  ^  ^  (  <j>(x0)  -  cf>{x  j(f  ))  |  d 


>  E 


9  (^o)' V^C^oI3-10) 


as  t  — ►  0.  Since  t  is  small, 


0  with  probability  l-Xrf  t  +o  {t ) 
Nj(t)=  1  with  probability  t  +o  (t ) 

>  2  with  probability  o(t). 


(3.11) 


Thus, 


E 


u  (x1(t))-ud  (yx  (t)) 


t 


\d  t  +o  (t) 


E  {ud  (x1(t)yud  (x1(t)+hd  (2/Io(rf)))  |  tx  <  t  }  + 


o(t) 


[ud(x 0)  -  «  (®0+/r(a:0))] 


(3.12) 
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as  t  |0.  In  addition 


E  f  d(yXo(s))  =  (l-\ds+o  (s))E  {f  d(x1(s))\r1>s} 
+  (\ds  +o  (s))  .  E  {/  d  (x  i(s)+hd  (yXo(T{)))  |  tx<s  } 
v  +  o  (s  )  .  (high  order  jumps). 


Thus, 

t 

Urn  E  ~  J  f  ( yXQ(s  ),d)e  ~ Ps  ds  =  f  (xQ,d), 

t-+0  t  0 

so  that  the  right  hand  side  of  (3.8)  tends  to 

/  d(x0)  -  (Sud(x0). 

From  (2.5),  (3.8)  -  (3.15),  we  know  that  (3.5)  holds. 

On  the  other  hand,  if  ud  -  <f>  attains  a  local  minimum  at  z0  E  Htn  for 

then 

ud(z0)  ~  <t>(z0)  <  ud{z)  -  <j>(z) 
for  all  z  in  some  ball  B(z0,c)  with  e  small.  If 

ud(z0)  =  Mi  [«  ](*o), 

(3.6)  follows,  otherwise,  by  Theorem  2.7  for  t  small  enough, 


E 


ud(z0)  -  ud(yZQ(t)) 
— 


=  E 


t 

~  I  f  iVz0(s),d)e  ~ps  ds 
1  0 


+ 


«(&„(*)) 


(3.13) 


(3.14) 


(3.15) 


some  d , 


(3.16) 


(3.17) 
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In  the  same  way,  let 


*i(t)  =  z0  +  f  g  (y2o(s),d)ds 


then  the  right  hand  side  of  (3.17)  tends  to 


fd(z0)  -  fiud{z o) 


(3.18) 


as  t  |  0.  In  the  same  manner, 


E  {  U  {Zo)  °  ]  <  E  (  ^*o)  - 


(3.19) 


-  g  (z0)'V<f>(z o) 


and 


E 


«  (*i (t))  -  ud{yz(t)) 


t 


\d[ud{z0)  -  ud(z0+hd(z0))}  (3.20) 


as  f  J,  0.  In  view  of  (3.17)  -  (3.20),  we  know  that  (3.6)  holds.  Thus,  u  is  a  viscosity 
solution  of  the  (QVI). 


QED 

Before  showing  the  existence  of  a  solution  to  the  (QVI),  we  show  that  (3.3)  admits  a 
unique  solution,  so  that  any  functions  constructed  to  satisfy  (3.3)  must  be  the  optimal 
solution. 

Lemma  3.2.  If  u  —  (u1,  .  .  .  ,  um )  is  any  viscosity  solution  of  (3.3),  then 

u  d  (x  )  <  Md  [u  }(x  ),  V/  x  E  IRn  ,  d  E  {l  (3.21) 
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Proof.  Suppose  =t  x0  G  IRn  such  that  (3.21)  does  not  hold.  Then  for 
u  €  C0R")m, 

ud(x)  >  ud(x)  +  k(d,d)  (3.22) 

for  all  x  G  B(x0,e),  e  small  and  some  d  ^  d.  It  is  not  hard  to  show  that  there  exists 
a  smooth  function  cf)  G  (71(IRn  )  such  that  ud  -  f>  attains  a  local  maximum  at  some  point 
x1  G  B(x0,e).  Thus,  by  the  definition  of  viscosity  solution, 

max  {  /3u  d  (x  J  -  gd(  x  f)-sj  f)(x  f)  -  \d  [u  d  (x  d  (x  J)  -  ui{x1)\  -  fd(x1), 

ui(x1)  -  Md  [w  ](ar  j)  }  <  0. 

In  particular,  ud{xl)  <  Md  [«](a;1)  which  contradicts  (3.22). 


QED 

Theorem  3.3.  If  u  —  (u1,  .  .  .  ,  um  )  and  v  —  (v1,  .  .  .  ,  vm)  are  viscosity  solu¬ 
tions  of  (3.3).  Then  u  =  v  . 

Proof.  Since  u  ,v  G  BUG  (1R”  )m  ,  let 

K  =  max  { |  |w  |  [,  |  |n  |  |,  1  }  <  oo. 

Choose  7  G  C'1(IRn)  such  that 

7(0)  =  5 K  ,  |  V7  |  <  10 K 

0  <  7(3; )  <  5 K  if  x  7^  0  (3.23) 

7 (x )  =  0  if  |  x  |  >  1. 

X 

and  let  ■yfx )  =  7( — ),  e  >  0,  x  G  IRn  •  Consider  the  auxiliary  function 
e 

*d{x,y)  =  ud(x)  -  vd(y)  +  7 e(x~y)-  (3-24) 

Since  u  ,v  and  7£  are  bounded,  then  for  each  e  >  0,  =)  [x  vy  f]  G  IR2,!  such  that 
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e. 


(3.25) 


max  <frd  (x  lty  y)  >  sup  max  3>d(x,y)  - 
x  ,y  1  <rf<m 

Since  the  supremum  in  (3.25)  may  not  be  attained  at  some  point,  we  have  to  add  an 
additional  small  function  to  make  it  occur.  Fix  e  >  0  and  (x1,y1),  choose 

£  E  C1(IR2"  )  such  that 

'  a^vVi)  =  1.  0  <  £  <  l,  |  v£|  <2 

.  €(x,y)<  1  if  (x  ,y  )  (x  vy  i)  (3.26) 

$(z  >y)  =  0  if  I  I-Ij  I  2  +  |  z/  —2/ 1  |  2  >  1. 

Define 

(x  ,y )  =  $>d  (x,y)  +  2e£(x  ,y  ) 

=  ud(x )  -  vd(y)  +  7  e(x-y)  +  2e£(x  ,y). 

Then  for  any  d ,  (x2,y 2)  such  that  |  x2-x1  |  2  +  \  y2-yx  |  2  >  1,  we  have  l~(x2,y 2)  =  0 

and 

(x 2,y  2)  —  (x  2,y  2)  <  s  up  max  <f >d(x,y). 

x  ,y  1  <d  <  m 

But  by  (3.25), 

max  '&d(Xi,yi)~  max  <frd(xl,yl)  +  2e  >  sup  max  <t>d  (x  ,y )  +  t, 

l<d<m  1  <d  <m  x,yl<d<m 

so  that  =4  ( x0,y0 )  (E  IR7n  with 

I  x0-x  j  2  f  |  2/ o  2/  i  |  2  <  1  (3.27) 

and  such  that 

^(Zcvlfo)  =  max  max  (x  ,y).  (3.28) 

We  claim  that  |  x0  -  y0  |  =  o  (e)  as  e  |  0.  Note  that  suppose  |x0  -  y0\  >  e.  Then, 
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*d(Zo,yo)  =  ud{x0)  -  vd(y0)  +  7t{x0-y0)  +  2 e£(x0,y0) 


<  2 K  +  2e  <  3 K 

where  we  can  assume  2e  <  K  without  loss  of  generality.  But, 

'\>i  (x  ,x)  —  ud  {x)  -  vd(x )  +  7e(0)  +  2e!-(x,x) 

>  -  K  -  K  +5  K  =  3  K 

which  is  a  contradiction,  so  that  |  x0  -  y0  j  <  e.  We  will  refine  the  above  estimate  as 
follows.  Since  't>d  attains  a  maximum  at  (x0,y 0),  then 

ud(x0)  -  vd(y0)  +  -jt{x0-y0)  +  2e£(x 0,y0) 

=  *d(x0,y0) 

>  Vd(x0,x0) 

=  ud{x0)  -  vd(x0 )  +  7e(0)  +  2ef(z 0,x0) 

which  implies 

lei^o-Vo)  >  7e(0)  +  vd(y 0)  -  vd(x0)  +  2e[£(:r0,z0)  -  £{x0>y0)\ 

=  7e(°)  +  0  (!)  as  e  I  0  (3.29) 

for  v  continuous,  |  x0  -  y0  |  <  e  and  0  <  £  <  1.  Thus,  from  (3.29) 


Xo-Vo 

lim  7  - 

e->o  e 


=  lim  lt(x0-y  0)  >  5 K 

e— >0 


and  so 


I xo  Vo  I 

lim  - -  =  0 

1 1  o  e 


i.e.,  |a0  -  i/0  |  =  o  (e)  as  t  {  0.  For  fixed  y0,  let 


<Pi(x)  4  vd(y 0)  -  ie(x~y0)  -  2 c£(x,y0). 


(3.30) 


31 


Then  <j)1  E  C'1(IRn)  and  tyd(x,y0)=  ud{x)  -  <7(z )  attains  its  maximum  at  x0,  so 
that 

max  {  (3u  ~d{x0)  -  gd  {x  0)-\/<t>x{x  0)  -  \}{ud(x0  +  hd(x0))  -  ud{x0)}  -  f  d(x 0), 

ud(x0)  -  Md[u){x0)}  <  0.  (3.31) 

In  the  same  way,  for  fixed  x0,  let 

<t>2(V  )  =  ud(x0)  +  7£(z0 -y)  +  2 e£(x0,y).  (3.32) 

Then  (j)2  E  C1(IR’i)  and  -  ^d(x0,y )  =  v d  (y )  -  (j)2(y)  attains  its  minimum  at  y0. 
Thus, 

max  {  Pv  d(y0)  -  g  d(y0)- V^iVo)  -  K  [t>  d(y0  +  h  d(y0))  ~  vd{y0)}  -  fd(y0), 

vd(y0)  ~  Md[v](y0)}  >  0.  (3.33) 

We  consider  the  following  two  cases. 

Case  (i)  vd(y0)  <  Md[v}(y0).  Then  from  (3.31) 

(3ud{x0)  <  ^(zoVV^iOo)  +  K[ud(x0+hd(x0))  -  ud(x0)}  +  f'd{x0) 

—  gd (xo)~  ~  7  V7(  X\y°  ~  ztVx&o.yo) 

+  \j[ud(x0+hd(x0))  -  ud(x0)\  +  f  d(x0).  (3.34) 

By  (3.33),  we  have 

0v‘d(yo)  >  gd(yo)-v<t>2(yo)  +  ^d[vd(yo+hd(y0))  -  vd(y0)}  +  f  d(y0) 

~d  f  \  1  {  xo~Vo  )  c,  . 

=  g  (vo)'  V7  -  +  2€v„  £(*0.yo) 

eye) 

+  Kivd(yo+h'd(y0))  -  vd(y0)}  +  f  d {y „).  (3.35) 

From  (3.34)  and  (3.35),  we  get 
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/3{ud{x0)  -  vd{y0)] 


<  -  j[gd(x  o)  -  gd(yo)]-vi  |  x-°~- }  -  2e[sy x  t(x 0,y 0)  +  v#£(*o.yo)] 

+  ^dlu  ~d(xo+h  ~d(xo))  ~  «‘<(*o)  -  v  'd (y0+h  d(y0))  +  vd(y0)} 

+  /  d(xo)  -  f  d(y0)-  (3.36) 


Note  that,  since  |  x0-y0  |  =  o  (e),  f  and  g  are  Lipschitz  continuous,  we  have 


\fd(xo)  -  fd(y 0)l  =0(1) 


V7 


x  0-2/  o 


9d(x o)  -  0rf(S/o) 


<  10/a- 


*cri/o 


(3.37) 


o(l)  (3.38) 


as  e  |  0  and 


2e(  |  V.£(*0.y0)  I  +  I  V»f(*0.y0)  |  )  <  8e. 


(3.39) 


Moreover, 


=  ud(x0+hd(x0))  -  va(y0+ha(x0))  +  it(x0-y0)  +  2  e^(x  0+h  “  (x0),y0+h  d  (x  0)). 


«  (*o)  -  v  (2/0)  +  le(xo~Vo)  +  2e£(*o>2/o) 
=  ^(*0.2/0) 

>  ^^(a;0+/i^(a;0),?/0  +  hd(x0)) 

,d/„,  ,  l  d 


Thus, 

«^o)  -  ■“<J(*o+/j<i(*o))  -  orf(y0)  +  Vd(y0+hd{y0)) 

>  v'd(yo+hd(y0))  -  vd(y0+hd(x0))  +  2 e[£(x0+hd{x0)ty0  +  hd(x0))  -  £(x0,y0)} 

=  o(l)  as  e  |  0.  (3.40) 

From  (3.36)  -  (3.40),  we  know  that 


«  (*o)  -  v  (2/o)  <  0  (!) 


(3.41) 


Again, 
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which  implies 


ud{xQ)  -  vd(y0 )  +  ~u(x0~y0)  +  2e£{x0,y0) 

=  o,2/o) 

>  'i>d  {x  ,X  ) 

—  ud(x)  -  Vd(x)  +  %(0)  +  2€^(x,x) 


ud  (x)  -  v  d  (x  ) 

<  [«^(*o)  -  u^(2/o)]  +  l7t(* o-3/o)  -  7e(°)J  +  2e[?(x0>t/0)  -  £(*,*)] 

—  0(1)  as  e  |  0. 

Thus,  ud(x)  <  a (a )  for  all  x  €  IRn  and  d  £  If  we  change  Che  role  of  w  and 

v  in  our  argument,  we  have  vd{x)  <  ud{x).  Hence,  u  =  v . 

Case  (ii)  vd(y0)  —  Md  [v  ](y0).  Then  zj  d  =^=  d  such  that 

v\y0)  =  vd{y0)  +  k(d,d).  (3.42) 

Then, 

ud(x0)  -  vd{y 0)  +  7£(*o~2/o)  +  2e€(x a>Vo ) 

—  yd(xo,yo) 

>  tyi{x0,y0) 

=  ud{x0)  -  vd(y 0)  7-  7t(x0,y0)  +  2e£{xQ,y0) 

which  implies 

ud(x0)  ~  ud{x0)  >  vd{y0)  -  vd(y0)  =  k(d,d). 

But  in  general, 

ud(x0)  <  ud(xQ)  +  k(d,d), 

so  that 
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ud  Oo)  -  ud{x0)  =  Vd(y0)  -  vd(y  0) 

and 

'J^Oo.yo)  =  (x  o>V  o)- 

Now,  we  can  consider  the  same  situation  with  index  d  instead  of  index  d  .  If  there  exists 
d  d  such  that 

vd(y0 )  =  vd  (y0)  +  k{d  ,d). 

From  (3.42), 

vd{yQ)  —  V  J(y0)  +  k(d,d )  -F  k(d,d) 

>  vd(y0)  +  k(~d  ,d) 

which  contradicts 

v'd(y0)  <  '^(f/o)  +  k(d,d),  V  d  d . 

Hence,  we  are  in  case  (i)  with  index  d  instead  of  index  d  and  the  proof  is  completed. 

QED 


4.  Existence  of  Viscosity  Solutions. 

Now,  we  use  a  finite  difference  approximation  to  construct  a  sequence  of  solutions 
which  converges  to  the  solution  of  (3.3). 

Let  p  £  C'2(]R"  )  such  that 

p(x  )  =  0,  x  <  0 

•  p{x )  >  0,  x  >  0  (4.1) 

.  0  <  p'  (x)  <  1,  p' '  (x )  >  0  for  x  >  0 
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and  pt{x )  =  p(-),  x  £  IRn  ,  e  >  0. 
e 

Consider  the  penalized  system  of  approximation. 


0uf(x)  -  —  [ud{x+egd  {x))-ud{x)\  -  \d[ud  (x +hd  (x))-uf  (x)} 
€ 

+  E  Pe(ut(x)~uf  (x)-k(dj))  =  f  d(x) 


(4.2) 


or 

udt{x)  -  [ud(x+egd(x))-udt(x)\  -  — |-[u  \  (x  +h  d  {x  ))-«  f  (x  )] 

/3e  p 

+  JjrE  Pt(ui(x)~ui  (xhk(d  ,d))  =  jjf  d  (x).  (43-) 

dj&d 

We  define  operators  A,!!^  fl2:  C(IR.”)m  — ►  C'(1R")”!  such  that  Au  = 


(aE,  . 

.  . ,  Am  u ),  riju  =  (n/w ,  . 

.  .  ,  n1mw )  and  n2u  =  (n2u ,  .  .  . 

,  n2mw )  where 

Ad  u(x)  = 

[ud(x+tgd(x))  -  ud(x )] 

(4.4) 

n  du(x)  ~ 

-  -jj-  [«  d  (x  +h  d  (X  ))  -  Ud(x)\ 

(4.5) 

n 2du(x)  t 

~  E  Pe(U  d  (*  )~U  d(X  )~k  <d  ))• 

(4.6) 

Definition,  (i)  An  operator  S  :X  —y  X  with  domain  D(S)  is  said  to  be  accretive 
on  the  real  Banach  space  X  if 

\\x  -  x  +  7[S'  ( x  )-S  (x )]  1 1  >  |  \x  -  x  1 1  (4.7) 

for  all  x ,  x  E  D  {S),  V  7  >  0. 

(ii)  An  operator  S  is  said  to  be  m  —accretive  on  X  if  S  is  accretive  on  X  and  the 
range  R  (/+  7S)  =  X  for  all  7  >  0  (or  equivalently  for  some  7  >  0). 
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The  following  lemma  is  from  Evans  [9,  pp.  242]. 

Perturbation  Lemma  4.1.  If  S  is  m-accretive  on  X=^C{tRn)m  and  T  is  accre¬ 
tive,  Lipschitz  continuous  everywhere  defined  on  X,  then  (S+T)  is  m-accretive  on  X,  in 
particular,  the  range  R  (/  +5  +  T )  —  G  (IRn  )m  . 


Lemma  4.2.  A  is  m-accretive  on  C  (IRn  )m  . 

Proof.  Suppose  there  exist  x0  and  d  such  that 

ud(x0)  -  ud{x 0)  =  max  sup  |t/rf(a;)  -  ud(x)  j  ~  ||u  -  u 

^  16E' 


(4.8) 


then 


Ad  u(  x0)  -  A  d  u  ( x0 ) 

=  ~  ~  +  eg'd(x0))  -  ud(x o)]  +  -±-  [ud(x0  +  egd(x 0))  -  u\x0)] 

=  -^  {{ud(x0)  -  ud{x0)\  -  [u~d(x0  +  egd(x0))  -  ud(x0+egd(x0))}} 

=  { I  lM  “  U  II  -  +£9d(x0))  ~  ud(x0  +  egd(x0))}} 

>  0 


so  that 

||w-u  +7(Am-Aw)||  >  u  d{x0)~u  d{x0)  +  7[Ad  u  d(x0)-Ad  u  d{x0)] 

>  u  d(x0)  -  u  d{xQ) 

=  |  \u  -  u  |  |. 

If  there  exists  no  x0  such  that  (4.8)  holds,  then  for  each  e  >  0,  let  x e,  d t  be  such  that 

ud\x  £)  -  ud\x  t)  >  1 1  u  -  m  1 1  -  e 
Choose  C^IR.")  such  that 
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SO*e)  =  1 

0  <  ^(ar )  <  1  for  x  ^  xt  (4-9) 

f(x)  =  0  for  1 1  a:  — rr  e  j  |  >  1. 

Consider  the  auxiliary  function 

Od(x)  ~  ud(x)  -  ud(x )  +  2e$(x).  (4.10) 

Then, 

<t>de(x  e)  =  ud‘(  xe)  -  ud\xt)  +  2e  >  ||u-u||  +  e 

and 

<&d(x)  =  ud(x)  -  ud(x)<  |  |u  — u  || 

if  x  (£  B  {x  v  1),  so  that  there  is  a  dl  such  that  <frd'  attains  a  maximum  at  xv  say,  in  the 
ball  B  (a: €,  1).  Thus, 

Arfl«(*!)  -  A  ilu(x1)  +  -  g(x1+egd\x  i))] 

=  +  26^!)] 

-  [u</'(x1  +  egi\xl))  -  ^'(ii +  ^Vi))  +  2etfx  j  +  eg  d \x  J)]} 

>  0 


so  that 


1 1 u  ~u  +  7(Au  -Am  )  1 1 

>  u  rfl(Xi)-«  d\x1)  +  7[Arflu  (x1)-Adlu  (xj] 

>  ®d\xi)  ~  +  ~~[<&x1+egdl(x1))  -  tfxj] 
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>  1 14>  1 1  +  o  (1)  as  e  |  0  since  £  £  C 1 
>  ude(xt)  -  ud\xe)  +  2e^xe)  +  o(  1)  asejO 
>  \\u-u  \  \  -  e  +  2eq(xt)  +  o{  1)  as  e  j  0 
=  |  \u  -  u  1 1  +  o  (1)  as  e  J  0. 

Letting  e  |  0,  we  show  that  A  is  accretive.  To  show  R  (1+  A)  =  <7(IR,!  )m  ,  i.e.,  the  equa¬ 
tion  u  +  Au  =  q  is  solvable  for  all  q  £  C  (HR”  )m .  Then, 

ud{x)  -  -j-  [u  d  (x  +eg  d  (x  ))  -  u  d  {x  )]  =  q  d  (x  ) 

or 


■u  d  (x  )  = 


1  +/3e 


[w  (x+eg  (x))  +  /3eqd(x)} 


=  Tud(x), 


and 


Tud{x)  -  Tvd(x) 


!+/?€ 


[ud  {x+egd  (x))  -  vd(x+egd(x))} 


< 


l+/3e 


I \u 


d  -  vd 


which  implies 


I Tud  -  Tvd  II  < 


l+(3e 


in"  -  »'ll 


so  T  is  a  contraction.  By  the  contraction  mapping  theorem,  there  exists  a  unique  fixed 
solution  ud  £  £7(IR”).  Hence,  A  is  m-accretive. 


QED 
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Remark.  In  the  same  manner,  we  can  show  is  m-accretive. 

Lemma  4.2.  n2  is  accretive. 

Proof.  Define 

Ad’du(x)  =  ud(x)  -  ud{x)  -  k(d,d). 

Suppose  z0  and  d  such  that 

ud{x0)  -  u  ~d(x0)  =  max  sup  |  u  d  (x  )  -  u  d  (x  )  |  £  ||u-u||.  (4  ,  n 

d  !  eiR« 

Then,  for  any  d  d , 

A dJn{xQ)  -  AiJu(x0 ) 

=  {ud{xQ)  -  ud(x0)  -  k(d~d)\  -  \ud{x0)  -  ud{x0)  -  k(d~d)} 

=  [ud(x0)  -  ud(xQ)}  -  [ud{x 0)  -  ud{x 0)] 

>  0. 

Since  pt  is  continuously  increasing,  we  have 

pt{Ad  ~d u  (x 0))  >  pt{AdJu  (x0)) 

and  so 

n  du(x0)  >  Yl2du(x0). 

Thus, 

II  u-u  +  7(n2w-n2M  )  1 1  >  ud{x0)  -  u  d(x0)  +  7[n/w(o:0)  -  ndu{x0)\ 

>  ud(x o)  -  u  d(x0) 

=  II  U~u  ||  (4.12) 

which  shows  that  n2  is  accretive.  Again,  if  there  exists  no  xQ  such  that  (4.11)  holds,  we 
can  consider  a  similar  auxiliary  function  as  (4.10)  to  show  (4.12). 
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Lemma  4.3.  11  =  11!  +  n2  is  Lipschitz  continuous  on  C  (IRn  )m  . 
Proof.  Since 


|  Tldu  (x  )  -  n/v  ( x  )  | 

=  — |  -  u  d  (x  -\-h  d  (a; ))  -f  ud(x)  +  vd(x+hd(x))  -  vrf(a;)| 
<  [\ud{x)  -  vd(x)  I  +  I  u  d  {x  +h  d  (x  ))  -  V  d  (x  +h  d  (x  ))  |  ] 

we  have 

l|ni«  -  nxv  ||  <  2~-  1 1 u  —v  ||. 

Again, 

|  U2du  (x  )  -  n2dv  (x  )  | 

=  -j  |  E  Pt{ud  (x)-ud(x)~k(d  ,d))  -  pt(vd  (x)-v  d(x)-k{d  ,d))  \ 

<  jj  E  IIp'cII  I  «rf(z)  -  vrf(a:)  -  wrf(a:)  +  v‘d(x)\ 

d^d 

--  2(wi-l)  I,  n 

pe 

so  that 


I  in2U 


n2u 


< 


2{m  -1) 


u  -v 


Thus, 

I |n«  -  n«  1 1  <  -|[Xrf  ||«-«|| 

which  shows  that  n  is  Lipschitz  continuous  on  C  (IRn  )m  . 
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QED 


By  the  Perturbation  Lemma  4.1,  A  +  II  is  m-accretive  and  so,  for  each  e  > 
have  a  solution  u  e  E  C(IRn  )m  of  (4.3). 


Theorem  4.4. 


(i) 

(ii) 


o  <  uf(X)  <  ILLii 

/5 

For  eac/i  0  <  7  <  min 


,  e  >  0,  d  El 
(  L  (l+Xmax)  ) 


\u${x)  -  uf{x)\  <  C1\x-x  |  7,  x  E  IR” ,  e  >  0,  d  E  {l,  .  .  .  ,  m 
with  the  same  constant  C 7  w  (2.20).  If  f3  >  L  (l+Xmax),  we  can  take  7=1. 

Proof,  (i)  Suppose  =|  x0  and  d  such  that 

u  f  (z0)  =  min  min  uf(x) 
d  1  eR" 

then 

uf  (z0)  <  uf  (x0)  +  k(d,d),  d  7^  d , 

so  that 

-j  S  Pe(uf  (xohuf  (x0yk(d,d))  =  0. 

d^d 


Thus  from  (4.3),  we  have 

u  't  (X0)  >  ±fd(x0)  >  0, 

P 


so  that 

uf  (x  )  >  u  f  (z0)  >  0. 


0,  we 


(4.13) 


(4.14) 
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If  there  exist  no  Xq ,  d  such  that  the  minimum  in  (4.13)  occurs,  we  can  consider 
ut(x)  -  ‘2e^ix)  instead  of  uf(x)  to  force  the  minimum  in  (4.13)  to  occur.  The  same 
analysis  can  be  adapted  from  the  proof  of  Lemma  4.2  to  show  (4.14). 

Without  loss  of  generality,  we  can  assume  there  exists  x  v  d  such  that 

uf  {x  x)  =  max  max  uf(x). 

d  1  £R* 


Then  from  (4.3),  we  have 


Thus, 


«?  (*i)  <  4 /'(*!)  <  ■ 

p  P 


0  <  udt(x)  <  ud  {x  i)  <  -Lh  1  . 


(ii)  Let 


$d(  ,  a  1  ud{x+y)  -  ud(x ) 

ii/  r 


y  I  t4  o, 


where  7  will  be  determined  later. 


Again,  without  loss  of  generality,  we  can  suppose  =4  d ,  x0,  y0  such  that 


4>d{x0,y0)  =  max  max  ( x  ,y) 

d  x  ,y 


and  ud(x0-Jry0)  -  ud(x0 )  >  0,  y0  7^  0.  Then 


(^O^Z/o)  (^0)  V  0)  ^  e  (^o)»  ^  7^  d, 


so  that 


/3£(we  (^0+2/0)  -  «e  (a:  o+yQ)-k  (d,d))  >  pe(ud  (x0)-«?  (x0)-k  (d  ,d )).  (4.15) 


Since 
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ue  d 0+2/ o)  -  ~~lut  (xo+yo  +  €9d(xo  +  Vo))  ~  u  t  (xo+y0)\ 

~  p  lu  e  do+2/o  +  h  do+J/o))  ~  ue  (xO  +  y  o))] 

E  Pe(ut  do+2/o)  -  ut  (x  o+V  q)  -  k  {d  >d ))  =  —  /  d(x0+y0)  (4  i6) 


«?  do)  -  do  +  e</'d0  +  2/0))  ~  u]{x0)} 

- J-  [«?  do  +  *'do))  -  «‘f  (*o)] 


4-  E  Pe(«?  do)  -  «?  do)  -  k(d~d))  =  \f  d{x0). 
p  .  .  p 


(4.17) 


Substracting  (4.17)  from  (4.16),  we  have 


U  +  +  "jl")  Iu«  do+2/o)  ~  do)l 


=  j  ^  'do+2/0)  -  /  'do)]  +  d 0+2/0  +  cs,rfd°  +  2/°))  -  do  +  €ff'd0))l 


X 


+  -J-  [«e  do+2/o  +  /Edo+2/o))  -  do  +  ^'do))l 


/? 


E  [Pe(«£  do+2/o)-<  do+Z/oM1  (<M))  -  Pe(«e  do)-«£  do)-*  (fif  ,</))] 


<  4  ^  'd°+y°)  -  /'do)]  +  4"  de  do+2/o  +  e</'do  4-  2/0))  -  do  +egd(x0))} 


X 


+  y  [“fdo+yo  +  ^do+l/o))  -  ui  (x0  +  hd(x0))}. 


(4.18) 


Dividing  (4.18)  by  |  y0  \  1,  we  have 


1  X  j 

(1V  +  T> 


«'  d  0+2/0)  -  do) 
1 2/0 1 7 
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<  j_  1  /  d(^0+yo)  -  /  j(j0)  I 
~  P  i  !/o  1 7 

I  u]  (x0+y0  +  egd(x0+y  0))  -  nf  (ar0  eff  ^  C^o))  |  [  Vo  +  tg  d{x0+yQ)-eg  d(x0)  |  7 

fit  I  2/o  +  e/^o+J/oH/^o)  I  7  |  2/o  I  7 

Xi  |  (a:o+y0+/^(a:o+2/o))  ~  « \  (x0+h  ^(ar0))  |  |  Vo+h  d(x0+y0)  -  hd(x0)\ 7 

P  I  y0+hd(xo~tyo)  -  hd(x0)  | 7  |  y0 1 7 

<  j  11/  ll1-7^7  +  -^<^(2:0,i/0)(l+eL)7  +  ~~*dt x0,y0){l+LV ■ 


Thus, 


{  1  +  ^[l-d  +tLV}  +  ~  [l-(l+L  )7]  }  *‘(x0,y0)  <~\\f  II1-7  (4.19) 


Let  the  constant  be 


a,  t  l  +  j-Nl  +eZ,)7]  +  ^-[1-(1+D7). 

Then  a7  is  a  decreasing  function  of  7  with  a0  =  1.  We  want  to  find  the  range  of  7  such 
that  a7  >  0.  We  claim  1+27  >  (I+2;)7  for  x  >  0,  0  <  7  <  1.  Let  ^(7)  =  (I+2:)7 
and  ^2(7 )  =  1+2:7.  Since  <d  is  a  convex  increasing  function  such  that 
<d(0)  =  1  =  02(O),  and  <d(l)  =  l+z  =  <f>2(  1),  <d( 7)  <  <^(7),  0  <  7  <  1.  Thus, 

1  -  (I+2;)7  >  -2:7,  V/  x  >  0,  0  <  7  <  1.  Hence 


a7  >  1  +  -7r(  -  eL  7)  +  -—(-Li) 

1  pe  P 

=  1  -  Li(i+xi) 


>0  if  7  <  min 


P 


L  (14~Xmax) 


,1 


With  this  choice  of  7,  we  have 
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QED 


*d(z,y)  I  <  *d(*0,yo)  <~~\\f  IP==  c,. 

jjd  ^ 


From  the  above  lemma,  {«£}  are  uniformly  bounded  and  uniformly  HBlder  con¬ 
tinuous.  Then  by  the  Arzela-Ascoli  Theorem,  there  exists  a  subsequence  e/  such  that 
Uf  — ►  ud  E  C(IRn)  for  all  d  E  1  The  convergence  is  uniform  on  each  compact 

subset  of  IR”  .  In  fact,  u  is  bounded  and  HBlder  continuous  with  the  same  HBlder 
exponent  7.  We  shall  prove  that  u  solves  the  (QVI)  in  the  viscosity  sense. 

Theorem  4.5.  u£(  — ►  u  locally  uniformly  in  IR”  and  u  solves  (3.3)  in  the  viscosity 
sense. 

Proof.  We  first  claim  that  ud  <  Md[u}.  Suppose  not,  that  is,  =1  d  7^  d ,  6  >  0 
and  an  open  ball  B  such  that 

ud(x)  >  ud(x)  +  k(d,d)  +  28,  x  E  B. 

As  w  f  — >  ud ,  ud  — *  ud  uniformly  on  B ,  we  get 

u  f(  (x  )  >  u  d  (x )  +  k  (d  ,d )  +  S,  x  E  B 

for  all  sufficiently  large  /.  Now,  let  <f>  E  C'1(IR")  and  without  loss  of  generality,  we  can 
assume  x0  E  B  is  a  strict  local  maximum  of  ud  -  <j>\  otherwise  we  can  add  a  small 
function  to  force  this  to  occur.  Since  — *  u  d  uniformly  on  B ,  ud  ~  <f>  also  attains  a 
strict  local  maximum  at  xt  E  B  for  l  large.  Since 

uf  {xt)  -  [«£(  (x,  +elgd(xl))  u  (i,)]  -  [u^  (x/  +  hd(xt))  -  w£(  (a:,)]. 

Pti  P 
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then 


+  J  S  P"  («;  (*/)-<  (X[yk(d,d))  =  Lfd{Xl), 

d^d 


Pet(S)  <  S  p£/  (<  (x,)-k{d,d)) 


d^d 


<  -  (xt)  +  ~  [uj  (xt  +eigd(x,))  -  «J  (a;,)] 


+  [«J  (*/  +hd(xi))  -  uj  (X[)\  +  fd{Xl) 


<  11/  II  +  —\<Kxt  +elgd(xl))  -  #*,)]  4-  X,(77| /irf(x()  P  -f  ||/ 


<  2 


+  1 1 V  0 1 1  lb  II  +  xrf<MMP< 


oo. 


(4.20) 


But  the  left  hand  side  of  (4.20)  tends  to  infinity  as  e;  — *•  0,  which  is  a  contradiction. 
Thus  ud  <  Md  [u], 

(i)  As  above,  suppose  ud  -  (f>  attains  a  strict  local  maximum  at  x 0  £  B  while 
«{  -  <t>  attains  a  strict  local  maximum  at  xt  E  B  and  xt  — +  x0.  Then,  by  (4.2), 


Putt  (*<)““  +  tlgd{xl))  -  u*  (a:,)] 


so  that 


-  Xrf  K,  (x; +/*  (x, ))  -  «erf  (a:,)]  <  /rf(x,) 


0ud  {xi) - [<f>(xi  +  tlgd{xl))  -  <j>(x[)\ 

Cl 


-  x</  [«£(  (*/  +/*  (*t ))  “  «“  (a:,)]  <  /  d  {x[ ). 


Letting  /  — ►  oo,  we  have 
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0ud(xo)  -  gd  (x0)-S7<f>(x  o)  -  X,  [ud  (x0+hd  (x0))  -  ud(x0)}  <  fd(x0) 


so  that  (3.5)  holds. 

(ii)  Let  E  C'1(IRn)  and  ud  -  ip  attains  a  strict  local  minimum  at  z0.  If 
u  d  ( z0 )  =  Md  [u  }(z o)  then  (3.6)  holds.  Otherwise  u  d  (z0)  <  Md  [u  ](,z0),  so  that 

ufi(z)<Md[uei)(z)  for  2  E  some  ball  B 


and  ud  -  ip  attains  a  strict  local  minimum  at  Z[ .  Then  we  can  assume  zl  — ►  z0  without 
loss  of  generality.  Since  ud  (z[ )  <  Md[uei]  (*,), 

Pe,(ui(zl)  -  M<i  [ut,KZ[))  =  0. 


By  (4.2), 

(zi) - —  [udt  (zi  +elgd{zl))  -  ud  (^)] 

-  x,  [«j  (*t+A '(*,))  - «;  u)i  =  /rf(^) 


so  that 

Pu  i,  (zl) - —  Wzi  +  etgd  (zi ))  -  Hzi )] 

-  [«J  (*/+A '(*,))  -  <  (zt) ]  >  /rf(*;). 

As  l  — ►  oo,  we  have 

/3ud{z0 )  -  gd  (z0)-\jip(z0)  -  \d  [ud  (z0+hd  (z0))  -  Md(z0)]  >  fd(zo)- 


Thus,  (3.6)  holds  at  20.  Consequently,  w  is  a  viscosity  solution  of  (QVI). 


QED 


48 


Remark.  In  general,  u  is  only  Holder  continuous.  If  we  know  u  has  some  regu¬ 
larity  properties,  say  u '  exists  in  some  neighborhood,  then  one  can  show  u  satisfies 
(3.3)  in  the  ordinary  sense.  The  point  is  that  the  derivative  of  u  is  not  continuous 
across  characteristic  curves. 

5.  The  Case  of  Vanishing  Switching  Costs. 

In  the  case  when  the  switching  costs  vanish  (k(d  ,d)  =  0  for  some  d  d  in  (1.4)), 
then  the  dynamics  may  be  switched  at  any  time  without  incurring  a  cost;  hence,  the 
minimum  cost  does  not  depend  on  the  initial  control.  That  is, 

u1  —  u2  =  ■  ■  ■  =  um  =  u  (5.1) 

If  we  follow  the  arguments  used  in  the  previous  sections,  we  can  show  that  u  is  bounded 
and  Hblder  continuous  with  the  same  Hdlder  constant  C7  used  in  Lemma  2.4.  In  u  were 
continuously  differentiable  on  IRn  ,  then  by  the  principle  of  dynamic  programming,  u 
would  be  (formally)  a  solution  of  the  Hamiltonian  -  Jacobi  -  Bellman  equation 

max  {/3u  -  gd  •  V«  -  [u(-  +  h d )-«]-/  d  }  =  0  (52) 

d  =  l . m  v  ’ 

on  IR”  .  However,  u  is  not  always  (71.  By  invoking  the  same  arguments  used  in  section 
4,  we  can  show  that  u  is  the  unique  viscosity  solution  of  (5.2)  in  the  following  sense: 

Definition  5.1.  A  bounded  and  continuous  function  u  on  1R”  is  a  viscosity  solution  of 
(5.2)  if  for  each  $  £  Cx(IRn  )  such  that 

(i)  if  u  —  <j>  attains  a  local  maximum  at  x0  £  IR”  ,  then 

max  {(3u{x 0)  -  ^(a;0)rf  -\/u{x0)  (5.3) 

d  =1 . m 

-  \d  \u{xQ  +  hd(x  0))  -  «(a:0)]  -  /  d  (x  0)  }  <  0 
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and 


(ii)  if  u  -  (f>  attains  a  local  minimum  at  x0  £  IRn,  then 

max  {(du{z 0)  -  g(z0)d  -y«W  (5  4) 

~  [u(z0  +  hd  (z0))  -  u{z0)}  -  fd(z0)}>  0 

We  now  prove  that  the  optimality  system  is  closed ;  that  is,  each  value  function 
corresponding  to  non-zero  switching  costs  will  converge  to  u  as  the  switching  costs  tend 
to  zero.  The  result  corresponds  to  a  similar  result  in  Capuzzo  Dolcetta  -  Evans  [4], 

Theorem  5.1.  Suppose  we  have  a  set  of  switching  costs  {  k  fd  ,d )  }  such  that 

k  e(d  ,d)  >  0  V  d  d  £  {  1,  •  •  ■  ,m  }  (5.5) 

kt(d,d)  <  kt(d,d)  +  ke(d,d),  d  ^  d  d 

For  each  e  >  0  let  u  e  =  (« }  ,  ■  ■  ■  ,u  ™ )  be  the  unique  viscosity  solution  of  the 
corresponding  QVI  with  switching  costs  {kfd,d)}  and  let  u  be  the  unique  viscosity 
solution  of  (5.2).  If  k  fd  ,d )  — >  0  as  e  — *  0  for  all  d  ,d  £  {  1,  •  •  •  ,m  },  then  uf  ■—>  u 
as  e  — >  0  for  all  d  £  {  1,  •  •  •  ,m  }. 

Proof.  Since  the  {  uf  }  are  bounded  and  uniformly  HSlder  continuous  with  the 
same  HOlder  constant  as  in  Lemma  2.4,  there  exists  a  subsequence  {  }  such  that 

uf,  — * ■  ud  (say)  locally,  uniformly  on  IR”  ,  V  d .  By  Lemma  3.2 

uf  <Md[uti]<uf  +  ke<(d,d),  d^d  (5.6) 

and  so,  ud  <  u  d  as  — *•  0.  Since  d  and  d  are  arbitrary,  we  can  obtain  the  reverse  ine¬ 

quality  by  reversing  the  roles  of  d  and  d .  Thus, 

u1  =  u2  =  •••  =  wm  =  u  (5.7) 
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It  remains  to  show  that  u  is  the  viscosity  solution. 

(i)  Suppose  <f>  €  G1(\Rn  )  and  u  -  <f>  attains  a  strict  local  maximum  at  x0.  Then  for 

each  d  (E  {  1  }  and  each  sufficiently  small  ei  >  0,  uf  -  <j>  attains  a  local  max¬ 

imum  at  near  x0,  and  xf  — *  x0  an  — *•  0.  Since  ue  is  the  viscosity  solution 
corresponding  to  the  switching  costs  {  kf(d  ,d)  },  we  have 

(xi  )  “  )  '  V<Kxft  )  (5.8) 

-  X,  [ uc/Otf  +  /»'(*£  ))  -  )]  -  /*(<  )}  <  0 

Letting  ei  — *-  0 

ftu  (x  o)  -  ffrf(*o)  '  V<?K*o)  -  Xrf  [«(*o  +  hd(xQ))  -  m  (ar0)]  -  /  d(z0)  }  <  0  (5.9) 
Thus,  (5.3)  holds. 

(ii)  If  u  -  <t>  has  a  strict  local  minimum  at  some  point  z0,  then  for  each  d  and  each 
e;  small  enough,  uf  -  <f>  attains  a  local  minimum  near  z0.  So  we  can  choose  c?,-  and 
such  that 

(«e‘  ~  <l>)(Zi)  =  min  min  (u  f  -  <t>){z)  (5.10) 

2  o  in’1  d 

and  zi  — *  z 0  as  — *  0.  Hence, 

(«e‘  )  <  («  f,  -  )  (5-11) 

We  have 

u£(  (zj  <  )  +  k  t  (</j  )  \f  d  di  (5.12) 

and  so 

«e*  (^)  <  [  u£(  ](*,-)  (5.13) 
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Thus, 


fat,  (*i)  ~  9d'(Zi)  ■  V<f>(*i) 

-  ^d,  +hd'(zi ))  -  #f/’(z,)]  -  /rf,U')}  <  0 


(5.14) 


Passing  to  the  limit  as  ei  — *■  0, 

/M^o)  -  9  d'(zo)  '  V<K^o)  -  >N,  [«  Uo  +  ^'^o))  -  u  (z o))  -  /  ^ (*o)  }>  0(5.15) 


Thus,  (5.4)  holds. 

Since  (5.2)  has  a  unique  viscosity  solution,  the  limit  u  is  the  required  solution. 


QED 
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